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FOREWORD 


The  Job  Skills  Education  Program  (JSEP)  is  a  multi-phase  program  begun  in 
Fiscal  Year  1982,  and  designed  to  enhance  enlisted  career  potential  by 
improving  soldier  job  performance.  The  sponsor,  the  Education  Division, 

Office  of  the  Deputy  Chief  of  Staff  for  Personnel,  expects  JSEP  to  replace 
the  Army’s  current  Basic  Skills  Education  Program  when  it  is  implemented. 

The  JSEP  program,  being  developed  by  Florida  State  University  (FSU)  will 
result  in  a  standardized  curriculum  for  soldiers  who  demonstrate  deficiencies 
in  the  knowledge  and  skills  required  to  successfully  learn  their  Military 
Occupational  Specialty  (MOS) . 

In  accordance  with  current  policy,  JSEP  will  be  an  on-duty  program.  It  will 
also  use  a  computer-based  management  system  to  facilitate  an  open  entry/open 
exit  approach.  At  present,  most  of  the  lessons  being  developed  will  be 
computer  delivered;  however,  the  plan  calls  for  using  existing  materials,  and 
incorporating  materials  developed  as  part  of  other  AR1  efforts,  whenever 
appropriate. 

A  unique  aspect  of  JSEP  is  that  it  builds  upon  a  very  detailed  front-end 
analysis  of  MOS  Baseline  Skills.  The  analysis  covered  tasks  performed  by 
soldiers  in  the  94  highest  density  MOSs,  in  addition  to  Common  Tasks  (the 
skills  that  all  soldiers,  regardless  of  their  MOS,  need  to  know) .  Although  the 
Army  has  over  300  MOSs,  the  94  covered  in  the  analysis  represent  about  80%  of 
all  soldiers.  Perhaps  the  most  useful  product  developed  for  the  analysis  was 
a  taxonomy  listing  more  than  200  prerequisite  competencies. (P.C.)  for  these 
MOSs.  The  competencies  were  derived  from  detailed  reviews  of  Soldier  Manuals, 
and  from  extensive  interviews  with  subject-matter  experts  at  Army  schools. 

This  effort  produced  a  series  of  tests  intended  to  diagnose  deficiencies  in 
the  P.C.s.  Modified  versions  of  these  tests  will  be  used  in  JSEP. 

The  JSEP  program  will  include  a  front-end  learning  strategies  module 
designed  to  improve  soldier  skills  In  reading,  studying,  test  taking,  and 
problem  solving.  The  curriculum  will  consist  of  this  strategies-trainlng,  plus 
180  diagnostic  review  lessons,  and  120  skill  development  lessons,  which  are 
being  developed  for  the  PLATO  and  MicroTICCIT  computer  systems.  The  program  is 
being  tried  out  at  two  TRAD0C  sites  and  two  F0RSC0M  sites,  prior  to  an  Army-wide 
phased  implementation. 


EXECUTIVE  SUMMARY 


Requirement: 

To  develop  a  predictive  Cost  and  Training  Effectiveness  Analysis  (CTEA) 
model  for  the  Jobs  Skills  Education  Program  (JSEP). 


Procedure: 

The  contractor  conducted  a  review  of  existing  CTEA  models,  emphasizing 
those  designed  for  military  applications.  Areas  for  discussion  included: 

o  cost  estimation 

o  training  effectiveness  estimation 
o  non-quantifiable  aspects  of  training  effectiveness 
o  integration  of  cost  and  effectiveness  data 
o  alternative  system  configurations 


Findings: 

Based  on  the  review  of  existing  CTEA  models  and  the  wide  range  of  features 
in  JSEP,  we  developed  a  model  to  accommodate  JSEP's  unique  requirements.  The 
features  addressed  and  their  bases  include: 

o  the  self-paced  nature  of  JSEP,  which  rendered  assumptions  underlying 
some  of  the  existing  CTEA  models  inappropriate. 

o  the  open  entry,  open  access  characteristics  of  the  program,  which 
require  "open"  system  methods  as  opposed  to  "closed"  system 
ones. 

o  the  need  to  synthesize  CTEA  models  into  one  which  addressed 
JSEP's  characteristics. 

Because  of  the  need  for  data  not  now  available,  we  plan  to  conduct  the 
predictive  CTEA  after  the  Phase  II  tryout.  The  effort  to  date  has  produced  a 
CTEA  model  that  will  address  the  unique  applications  problems  identified  during 
the  analysis  and  model  development. 
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PREDICTIVE  COST  AND  TRAINING  EFFECTIVENESS  ANALYSIS 


OVERVIEW 


Operational  Problem 

It  is  not  news  that  soldiers  must  be  trained  to  do  their  jobs.  They  must 
be  trained  so  that  each  Army  job  is  performed  competently —regard! ess  of 
differences  in  ability  and  background  in  newly  entering  soldiers.  To  accept 
less  would  cause  many  mission  elements  to  fail. 

Moreover,  many  Army  jobs  are  increasingly  dependent  upon  the  soldier's 
ability  to  use  high  technology  and  the  ability  to  learn  new  technology  as  it 
develops.  Soldiers,  therefore,  need  more  than  training.  They  need  enough 
education  to  be  able  to  learn  subsequent  jobs,  to  become  eligible  for 
promotion,  and  ultimately,  to  provide  leadership  for  tomorrow's  Army. 

The  Job  Skills  Education  Program  (JSEP)  is  designed  to  provide  soldiers 
with  job-related  basic  skills  instruction  that  is  prerequisite  to  learning 
their  skill  level  1  and  2  job  tasks  during  their  first  duty  assignments.  Based 
on  an  extensive  job  analysis  of  94  of  the  Military  Occupational  Specialties 
(MOS)  which  contain  the  largest  proportion  of  soldiers  and  tasks  contained  in 
the  Soldier's  Manual  of  Common  Tasks,  JSEP  provides  functional  basic  skills 
instruction  in  MOS  specific  requirements. 

As  It  Is  conceptualized,  the  JSEP  curriculum  recognizes  that  the  vast 
majority  of  soldiers  will  have  been  exposed  to  similar  basic  skills  instruction 
before  entering  the  Army.  Many  entering  soldiers,  however,  will  not  have 
learned  those  basic  skills  well  enough,  or  will  not  remember  what  they  learned. 
To  help  soldiers  learn  better  and  remember  more,  JSEP  Incorporates 
straightforward  training  In  research-based  learning  strategies  that  are 
directly  aimed  at  improving  learning  and  retention. 


Research  Objective 

The  purpose  of  this  report  Is  to  predict  the  costs  of  raising  soldiers' 
educational  levels  to  a  certain  minimum  level,  that  is,  to  the  level  at  which 
they  will  be  capable  of  learning  and  performing  their  first  duty  assignment 
jobs.  Since  JSEP  Is  In  Its  formative  stages  and  this  Is  an  analytical  report, 
our  approach  Is  to  focus  on  the  most  Important  cost  and  effectiveness  aspects 
of  the  program  rather  than  details  which  will  differ  from  post  to  post  and 
general  features  applicable  to  all  educational  systems.  This  report  will 
provide  an  Indication  of  the  costs  and  effectiveness  of  various  configurations 
of  CBI  systems. 

In  addition,  however,  to  preparing  for  Task  18,  Conduct  Cost  and  Training 
Effectiveness  Analysis  (CTEA),  the  Florida  State  University  (FSU)  has  analyzed 
various  CTEA  methodologies  (see  Appendix  for  a  description  of  various  other 
models).  The  conceptual  framework  presented  here  will  be  operationalized  and 


tested  in  the  preliminary  and  full-scale  tryouts.  Refinements  will  then  be 
made  on  the  model  on  the  basis  of  the  data  analysis  and  availability  as  well  as 
practical  constraints  and  problems. 


Scope 

CTEA  is  defined  as  a  methodology  that  involves  the  documentation  of  the 
comparative  effectiveness  and  costs  of  alternative  training  systems  for 
attaining  specified  performance  objectives.  Cost  refers  to  the  dollar-value  of 
the  human  and  capital  resources  required  to  implement  and  operate  an 
instructional  system,  in  this  case,  a  computer-based  instructional  (CBI) 
system.  Training  effectiveness  refers  to  the  degree  to  which  program 
performance  objectives  are  achieved.  In  the  JSEP  context,  it  is  a  level  of 
performance  that  is  to  be  achieved  and  the  purpose  of  this  report  is  to 
indicate  the  costs  of  alternative  systems  which  will  lead  to  the  achievement  of 
that  level.  The  value  of  CTEA  is  that  it  involves  the  simultaneous 
consideration  of  performance  and  the  costs  of  achieving  ft. 

It  should  be  noted  that  this  report  is  not  intended  to  evaluate  JSEP's 
contribution  to  job  performance  In  first  duty  assignments.  At  the  second  In 
Process  Review,  the  Army  Research  Institute  (ARI)  decided  that  it  would 
independently  design  a  study  and  evaluate  JSEP  in  this  respect  (Phase  I  Report, 
page  27). 


TRAINING  EFFECTIVENESS  ANALYSIS 

The  Task  7  Report  is  an  evaluation  plan  of  JSEP  which  will  be  implemented 
and  reported  upon  in  the  Task  17  Report  after  the  full  scale  tryout.  Since 
these  reports  specifically  treat  the  educational  effectiveness  of  JSEP  in 
detail,  we  will  not  duplicate  those  efforts  in  this  report.  Instead,  we  will 
use  some  relatively  easily  calculated,  easily  understood  indicators  of 
effectiveness  and  relate  them  to  the  costs  of  achieving  such  levels  of 
effectiveness. 

Several  indicators  of  the  training  effectiveness  of  JSEP  could  be 
calculated.  Some  are  "internal"  to  JSEP  while  others  are  "external."  By 
internal  we  mean  indicators  of  JSEP's  performance  with  respect  to  itself,  that 
is,  in  an  absolute  sense.  By  external,  we  mean  JSEP's  performance  in 
comparison  to  other  programs  (for  example,  BSEP)  and  measures  of  competency 
(such  as  GT  scores),  that  is.  In  a  relative  sense. 

The  Internal  Indicator  we  will  use  is  the  pass  rate,  that  is,  if  a  soldier 
passes  the  JSEP  Test,  he  will  be  considered  a  graduate  of  JSEP.  The  pass  rate 
is  an  indicator  of  the  quality  of  the  lessons.  If  it  is  low  the  lessons  can  be 
considered  too  difficult,  hard  to  understand,  unclear,  etc.  A  pass  rate  of 
about  80%  is  anticipated,  based  on  performance  from  the  preliminary  tryout  at 
Fort  Rucker,  Alabama. 
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Two  external  Indicators  suggest  themselves: 

1)  Changes  In  GT  scores  and 

2)  Changes  In  reenlistment  rates  or  numbers  qualifying  for 
reenlistment. 

Changes  In  GT  scores  offer  an  Immediate  Indication  « f  JSEP  effectiveness. 
Soldiers'  pre-JSEP  GT  scores  can  be  compared  to  their  post-JSEP  GT  achievement 
or,  alternatively,  JSEP  soldiers  can  be  compared  with  a  control  group  who  have 
not  been  in  the  program.  Changes  In  reenllstment  rates  or  numbers  of  soldiers 
qualifying  for  reenlistment  are  longer  term  indicators  of  JSEP  effectiveness 
and  much  more  difficult  to  attribute  accurately.  Reenlistment  rates  could  be 
dependent,  at  least  in  part,  on  the  job  market  for  younger  workers  and  the 
state  of  the  economy  in  general. 

To  answer  the  question  of  whether  JSEP  has  any  measurable  impact  on 
reenlistments  would  require  a  longitudinal  study  of  a  selected  JSEP  cohort  with 
appropriately  matched  controls.  We  believe  that  such  a  study  would  require 
tracking  soldiers  through  their  first  enlistment.  Therefore,  we  will  not 
analyze  JSEP's  Influence  on  reenlistments,  due  to  the  time  constraint. 

The  full  Impact  of  the  learner  strategies  instruction  will  not  be 
discernible  at  the  conclusion  of  the  Phase  III  trials.  While  pre-posttest 
differences  may  indicate  strategies  acquisition,  it  is  the  transfer  of  these 
strategies  to  the  job  situation  that  Is  critical.  However,  the  learning 
strategies  training  will  be  tested  In  that  this  training  will  not  be  given  to 
all  soldiers.  Differences  In  JSEP  test  scores  between  those  having  had  and 
those  in  the  control  group  not  having  had  the  training  will  p-ovide  a 
preliminary  Indication  of  its  Impact. 

For  these  reasons,  we  propose  to  concentrate  on  internal  measures  of  JSEP 
performance.  In  particular,  the  results  of  the  JSEP  Test  and  the  pass  rate. 

The  primary  Instrument  of  evaluation  to  be  used  at  the  Phase  III  trials  will  be 
the  JSEP  Test  that  Is  designed  to  measure  the  results  of  instruction.  It 
indicates  whether  soldiers  learned  the  material  assigned. 

The  pass  rate  Is  defined  as  the  ratio  of  the  number  of  JSEP  graduates  to 
the  number  of  JSEP  soldiers  (PR“  number  of  graduates/number  of  soldiers).  A 
graduate  Is  a  soldier  who  has  passed  the  JSEP  Test  for  his  curriculum. 

Soldiers  are  defined  as  those  soldiers  entering  JSEP  regardless  of  whether  they 
complete  It.  Some  adjustments  may  have  to  be  made  for  soldiers  who  are 
transferred  to  another  post  before  completing  the  program,  but  the  term 
"soldiers,"  In  general.  Is  meant  to  Include  dropouts,  those  falling  the  JSEP 
Test,  as  well  as  those  passing  the  JSEP  Test.  Additionally,  however,  changes 
In  GT  scores  will  be  calculated  as  an  external  indicator  of  JSEP  performance. 

The  statistics  described  above  are  effectiveness  Indicators.  In  addition, 
however,  the  Request  For  Proposal  requires  the  incorporation  of  costs  Into  the 
analysis.  The  primary  Indicator  of  both  cost  and  effectiveness  will  be  the 
average  cost  per  graduate  (AC/G).  The  AC/G  will  be  the  most  relevant  statistic 
of  the  analysis  because  It  Is  determined  by  factors  on  both  the  cost  and  the 
effectiveness  sides  of  the  program;  It  is  the  link  between  the  two.  The  other 
measures  are  relevant  to  one  or  the  other  but  not  to  both.  Since  the  AC/G  can 
vary  due  to  cost  as  well  as  to  effectiveness  variation,  as  a  comparison,  the 


average  cost  per  soldier  (  AC/S)  will  also  be  calculated.  Other  indicators  of 
cost-effectiveness  that  will  be  calculated  are:  average  cost  per  terminal  hour 
(AC/TH)  and  the  total  cost  of  each  system— PLATO  and  MicroTICCIT.  These  other 
measures  will  indicate  the  reason  AC/G  is  what  it  is.  For  example,  we  would 
like  to  find  a  low  AC/G  and  that  it  is  low  because  the  number  of  graduates  is 
high. 


While  the  AC/G  will  be  emphasized,  it  should  be  noted  that  there  are  other 
possible  sources  of  variation,  in  particular,  the  number  of  hours  a  soldier 
spends  in  JSEP  (which  can  impact  effectiveness)  and  the  number  of  workstations 
in  JSEP  (which  can  affect  costs).  In  both  cases,  however,  these  are  parameters 
within  which  we  will  work. 

Variation  in  the  number  of  JSEP  curriculum  hours  would  affect  the  number 
of  graduates  (and  the  pass  rate)  depending  on  the  change  in  the  amount  of 
material  to  be  learned  due  to  the  change  in  curriculum  hours. 

Increases  in  the  number  of  workstations  could  lead  to  economies  of  scale, 
that  is,  decreases  in  average  cost  due  to  an  increase  in  the  number  of 
workstations.  For  example,  a  single  host  processor  of  a  MicroTICCIT  system  can 
accommodate  up  to  64  workstations.  The  cost  of  a  host  processor  is  the  same 
regardless  of  the  number  of  terminals  linked  to  it;  therefore,  the  AC/terminal 
(and  implicitly  AC/G)  falls  as  the  number  of  terminals  is  increased  up  to  64. 
The  same  phenomenon  occurs  with  statistical  multiplexers  and  modems  in  the 
PLATO  system  (up  to  32  terminals).  We  will  indicate  where  such  potential 
economies  of  scale  may  occur. 

Throughout  the  analysis  we  will  assume  that  the  number  of  soldiers  in  JSEP 
and  the  number  of  hours  of  CBI  they  take  are  given  by  the  Army.  Approximately 
128  hours  of  CBI  will  be  taken  by  each  soldier  (the  exact  number  of  hours  may 
vary  somewhat  from  curriculum  to  curriculum).  In  addition,  the  amount  of  time 
required  to  complete  the  curriculum  will  vary  among  soldiers  due  to  JSEP's 
self-paced  nature.  The  number  of  soldiers  participating  in  the  full-scale 
tryout  will  be  limited  both  by  the  Army  as  well  as  by  the  amount  of  equipment 
available. 

Using  data  from  the  preliminary  and  full-scale  tryouts,  the  number  of 
graduates  will  be  statistically  predictable.  Furthermore,  life  cycle  costs  of 
the  systems  will  also  be  predictable.  The  ratio  of  the  present  value  of  the 
life  cycle  costs  of  the  system  to  the  number  of  graduates  is  another 
Interpretation  of  an  average  cost  per  graduate  that  is,  AC/G  ■  PVS/number  of 
graduates.  Alternatively,  costs  could  be  amortized  and  a  statistic  such  as 
*Total  cost  per  year/number  of  graduates  per  year"  could  be  computed  which 
would  also  be  Interpretable  as  average  cost  per  graduate.  Such  calculations 
are  not  Intended  to  generate  Identical  results,  instead,  they  are  meant  to  show 
that  a  statistic  can  be  interpreted  in  a  variety  of  equally  valid  ways. 

Costs  calculations  will  draw  heavily  on  the  Task  8  Report  (Cost-Benefit 
Tradeoff  Analysis).  In  addition,  all  statistics  will  be  given  for  both  PLATO 
and  MicroTICCIT.  Since  the  Instructional  content  will  be  comparable  on  both 
systems  (implying  identical  effectiveness),  cost  differences  become 
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significant.  However,  ARI  will  independently  investigate  whether  both  systems 
are  in  fact  equally  effective.  In  the  meantime,  however,  we  will  assume  both 
systems  deliver  the  same  product  and  that  the  relevant  question  is  which  system 
can  do  it  at  the  lowest  cost. 


RESOURCE  AND  COST  ANALYSIS 

JSEP  costs  can  be  separated  into  4  broad  categories: 

1)  Research  and  Development  (R  &  D)  Expenditures, 

2)  Capital  Equipment  and  Operating  Expenses, 

3)  Personnel  Requirements,  and 

4)  Facilities  Requirements. 


Research  and  Development  Expenditures 


Since  the  purpose  of  this  report  is  the  prediction  or  estimation  of  the 
most  cost  effective  configuration  of  JSEP,  R  &  D  Expenditures  should  not  be 
considered.  These  expenditures  are  sunk  costs,  that  is,  these  funds  have 
already  been  spent  and  are  unrecoupable.  Since  the  relevant  decision  is 
whether  to  spend  additional  money  on  JSEP,  for  example,  to  expand  it  Army-wide, 
it  is  immaterial  how  much  has  already  been  spent.  The  question  of  interest  is 
how  much  a  CBI  system  costs  to  operate;  therefore,  to  include  R  &  D 
Expenditures  would  merely  inflate  the  cost.  The  Army  already  owns  420  hours  of 
courseware  (though  it  is  still  being  developed)  and  can  use  it  as  it  wishes  at 
no  additional  cost.  The  cost  of  updating  and  revising  the  courseware  should  be 
included  as  an  operating  expense  but  not  the  entire  cost  of  developing  the 
program. 


An  alternative  approach  with  respect  to  R  &  D  expenditures  is  that  the 
Army  could  license  a  publishing  company  to  market  the  JSEP  materials.  Several 
educational  institutions  (for  example,  the  New  York  Commissioners  of  Education 
Office)  have  expressed  an  interest  In  adopting  or  adapting  JSEP  to  their 
systems.  Another  possibility  is  for  the  Army  to  simply  allow  any  institution 
desiring  use  of  the  courseware  to  have  access  to  it,  that  is,  provide  it  as  a 
public  service  or  for  public  relations  purposes. 


With  either  of  these  alternative  approaches  R  &  D  expenditures  could  be 
considered  to  pay  for  themselves  wholly  or  in  part.  It  is  immaterial,  however, 
for  JSEP  purposes  which  approach  Is  taken  since  the  funds  have  already  been 
spent. 


Capital  Equipment  and  Operating  Expenditures 


Capital  Equipment  and  Operating  Expenses  are  discussed  in  detail  in  the 
Task  8  Report  and,  therefore,  are  only  summarized  briefly  In  Tables  1  and  2. 

It  should  be  emphasized  that  It  Is  In  this  cost  category  that  differences  In 
costs  between  the  two  systems,  PLATO  and  MIcroTICCIT,  are  likely  to  arise. 

That  Is,  all  other  costs  of  JSEP,  such  as  facilities  and  personnel,  are 
essentially  the  same.  All  prices  listed  in  the  Tables  refer  to  single  units  of 
the  various  items;  the  numbers  in  parentheses  refer  to  the  number  of  units  of 


that  Item  required  for  the  full  scale  tryout  in  the  Spring,  1985.  Only  cost 
data  Is  given  for  Optional  Equipment  because  the  ratio  of  the  number  of  units 
of  such  equipment  to  the  number  of  terminals  Is  yet  to  be  decided. 


Table  1 

ESTIMATED  PLATO  COSTS 


Equipment  Costs: 

Viking  terminal—  COC  110  (32)  $3250 

Data  circuit  Installation  (2)  500 

Multiplexers  (4)  3710 

Modems  (4)  3674 

Equipment  cabinets  (2)  160 

Cables  (1/termlnal)  75 

(2/mux-modem)  50 

(1/printer)  50 

Annual  Expenses: 

Port  charges  and  libraries  (32)  3900 

Data  circuit  (2)  12000 

Maintenance  2000 

Optional  Equipment: 

Epson  FX80  Printer  595 

Digitized  Audio  3421 

Videotape  895 

Videodisc  5370 


Port  charges  and  libraries  are  the  rates  charged  by  the  FSU  Computer 
Center  (FSUCC)  for  PLATO  services.  Data  circuit  cost  estimates  are  based  on 
estimated  average  distances  from  the  central  computer  located  at  the  FSUCC  over 
commercial  communications  lines  at  commercial  rates.  The  Army  is  currently 
installing  a  PLATO  computer  (CYBER  825)  at  Ft.  Leavenworth.  The  Joint 
Committee  on  Computer  Based  Instruction  (JCCBI)  estimates  a  chargeback  of 
$500/term1nal /month  or  $6000  annually  per  terminal  for  the  use  of  Army 
facilities,  in  particular,  the  CYBER  at  Ft.  Leavenworth  and  Army  communications 
lines.  In  addition,  a  single  CYBER  825  can  accommodate  only  500-600  terminals; 
if  larger  systems  are  desired,  additional  central  computers  will  have  to  be 
purchased  (approximately  $600,000). 


Table  2 


ESTIMATED  MicroTICCIT  COSTS 


Capital  Equipment: 


Host  processor-IIIC  (2) 

$104600 

Terminal  (44) 

7400 

Annual  Expenses: 

Maintenance 

21140 

Optional  Equipment: 

Epson  FX100  printer 

1300 

Random-access  audio 

645 

Videotape 

500 

Videodisc 

5500 

Personnel  Requirements 

In  considering  Personnel  Requirements,  both  Instructor  and  soldier  costs 
should  be  considered.  In  particular,  JSEP  demands  on  both  will  be  given  in 
terms  of  their  monthly  salaries  and  benefits.  It  has  been  estimated  (Butman, 
p.13)  that  an  Instructor  In  a  CBI  system  could  accommodate  up  to  40  soldiers, 
however,  for  the  full-scale  tryout  JSEP  is  being  designed  with  an  Instructor- 
soldier  ratios  of  1:16  for  PLATO  and  1:20  for  MicroTICCIT.  These  instructor- 
soldier  ratios  were  determined  by  ARI  as  large  enough  to  adequately  test  JSEP 
In  the  full-scale  tryout.  There  is  nothing  Inherent  In  these  ratios  as  regards 
CBI  and  in  this  sense  they  are  essentially  arbitrary. 

Instructors  will  have  to  learn  to  teach  with  CBI  as  well  as  learn  to 
operate  the  systems.  That  is,  they  will  have  to  learn  certain  procedures,  for 
example,  starting  up  and  shutting  down  the  system,  however,  even  with  CBI  they 
are  an  Integral  part  of  the  system.  CBI  Is  not  meant  to  replace  them,  rather. 
It  Is  to  aid  them  In  doing  their  jobs  and  thereby  make  them  more  effective 
instructors.  The  Technology  Transfer  Plan  (Task  19)  will  deal  with  this  Issue 
at  length.  Instructors  manuals  will  be  prepared  explaining  the  procedures  for 
operating  the  systems.  For  the  full  scale  tryout,  one  week  will  be  allocated  to 
training  Instructors  In  the  use  of  the  systems. 

Soldiers'  time  Is  a  similar  matter;  assuming  soldiers  are  released  from 
their  regular  duties  to  attend  JSEP,  their  time  will  be  valued  at  their  gross 
monthly  salaries  and  estimated  benefits.  A  JSEP  curriculum  consists  of  128 
hours— approximately  equivalent  to  one  month  of  full  time  Instruction.  If 
soldiers  attend  JSEP  only  half  time  for  example,  their  salaries  and  benefits 
will  be  prorated  accordingly.  However,  soldiers  may  choose  to  spend 
additional,  off-duty  time  In  JSEP;  such  time  will  be  considered  free  to  the 
Army  and  not  valued  at  all. 

Valuing  soldiers'  time  Is  somewhat  arbitrary  In  that  some  soldiers  are 
presumably  very  competent  and  their  absence  from  the  job  will  be  a  burden  to 
their  supervisors  and  co-workers.  In  addition,  some  soldiers.  If  they  were  not 
In  JSEP,  would  be  in  BSEP  or  some  other  program  In  which  case  JSEP  would  not  be 


Incurring  any  additional  costs.  However,  with  these  caveats,  the  value  of 
soldiers'  time  will  be  estimated  according  to  their  monthly  earnings  plus 
benefits. 
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Facilities  Requirements 

It  was  anticipated  that  no  new  facilities,  for  example,  classrooms  or 
lighting  will  be  needed  for  the  full  scale  tryout,  that  JSEP  would  merely 
replace  BSEP  with  respect  to  classroom  space.  However,  additional  facilities 
may  have  to  be  constructed  at  Fort  Lewis,  Washington  due  to  a  lack  of  classroom 
space.  Otherwise,  JSEP  requirements  In  terms  of  wiring  and  electrical  outlets 
have  alreac(y  been  described  In  the  Task  5  Report  and  will  not  be  detailed 
further. 


ALTERNATIVE  CONFIGURATIONS 

PLATO  and  MIcroTICCIT  can  be  configured  in  a  variety  of  comparable  ways, 
therefore,  the  discussion  of  alternative  configurations  will  be  In  terms  of  a 
system  since  the  essential  differences  are  In  terms  of  costs  which  are 
documented  in  the  Task  8  Report  and  summarized  above.  The  basic  system 
consists  of  a  workstation  (primarily  the  terminal)  linked  to  a  computer 
providing  the  courseware.  To  this  basic  system,  several  optional  pieces  of 
equipment  such  as  a  printer  or  videodisc  can  be  added. 
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The  Task  5  Report  Is  Instructive  with  respect  to  the  effectiveness  of  the 
options  or  add-ons.  The  Task  5  Report  contains  an  analysis  of  184  PCs.  Of 
these,  21  were  not  deemed  feasible  for  CBI,  of  the  remaining  163,  128  were 
considered  teachable  with  CBI  alone— the  rest  could  be  enhanced  by  various 
auxiliary  devices  in  addition  to  the  CBI. 

These  optional  devices  are: 

1)  Printers,  handouts,  paper  and  pencil  assignments— 19  PCs, 

2)  Video  capability  (Interactive  and  passive)— 13  PCs, 

3)  Audio  capability— 3  PCs, 

4)  Hand-held  tutor— 2  PCs, 

5)  Calculator— 1  PC, 

6)  Instructors,  objects,  or  equipment— 20  PCs. 


Numbers  3  and  4  are  Immediately  suspect  with  respect  to  cost 
effectiveness.  Audio  Is  required  for  such  things  as  following  Instructions 
presented  from  multiple  sources;  tutors  are  to  be  used  to  practice  commonly 
misspelled  words.  Number  5,  a  calculator.  Is  needed  for  arithmetical 
operations,  however,  both  PLATO  and  MIcroTICCIT  have  calculators  programmed 
Into  the  systems  which  can  be  accessed  by  any  soldier  In  JSEP. 

Of  the  other  three  options.  Instructors  were  never  considered  optional  and 
little  or  no  justification  Is  required.  Printers  will  probably  prove  to  be 
cost  effective,  especially  In  light  of  the  fact  that  the  19  PCs  noted  above  are 
those  that  require  a  printer  specifically.  In  addition.  It  Is  likely  that  many 
other  PCs  would  benefit  from  the  availability  of  printers  (as  Is  Implied  In  the 
RFP  which  specifically  requires  a  hard  copy  capability).  Furthermore,  a  hard 


copy  capability  would  facilitate  the  soldier  management  system.  Video  Is 
somewhat  more  complicated  In  that  the  13  PCs  Include  both  passive  and 
Interactive  video. 

The  ratios  of  the  number  of  terminals  per  optional  device  has  not  been 
decided.  For  example,  considerable  capacity  redundancy  would  result  If 
printers  were  purchased  on  a  one-to-one  basis  with  terminals.  Furthermore, 
some  options  need  not  be  attached  or  Interfaced  with  a  terminal  at  all. 
Videotape  players,  for  example,  can  stand  alone  In  a  separate  workstation  and  a 
soldier  could  simply  take  a  videotape  to  it,  sit  and  watch  It. 

The  Delphi  or  expert  opinion  method  was  used  to  determine  the  technology 
best  suited  to  teach  the  various  PCs.  Additionally,  the  same  method  will  be 
used  to  judge  the  efficacy  of  a  video  capability. 


NON -QUANTIFIABLE  FACTORS 

Part  of  learning  strategies  training  Is  expressly  designed  to  be 
demonstrated  with  video  or  audio  media.  The  mood  management  module,  for 
example.  Is  very  dependent  on  video  in  that  the  techniques  are  demonstrated  by 
actors  in  “real  life"  situations.  The  actor-instructor  shows  the  soldier  how 
to  alter  his  mood  and  serves  as  a  role  model  for  him.  Such  a  technique  has 
been  shown  (Gage  and  Berliner,  chapter  15)  to  be  more  efficacious  than  mere 
description  or  statement. 

The  benefits  of  the  learning  strategies  training  are  relatively  long  term 
in  that  they  will  occur  only  after  the  soldier  is  back  on  the  job.  An  Interim 
test  for  the  effects  of  the  learning  strategies  Is  that.  In  the  full  scale 
tryout,  some  soldiers  will  be  given  the  training  while  others  will  be  given 
JSEP  instructional  materials  without,  however,  the  learning  strategies 
component. 

In  some  cases,  new  equipment  will  not  have  to  be  purchased  since  many 
Education  Centers  already  have  such  items  as  cassette  players.  Video  equipment 
Is  also  commonplace  and  may  not  have  to  be  purchased  specifically  for  JSEP. 

The  costs  of  producing  videotapes,  videodiscs,  and  audiotapes  depend  on  the 
specifics  of  the  scripts. 


DATA  COLLECTION 

Training  effectiveness  data  will  be  collected  as  described  In  the  Task  7 
Report.  It  will  be  a  relatively  simple  matter  to  obtain  numbers  of  soldiers, 
graduates,  and  pass  rates  from  the  course  management  systems. 

Cost  data,  however,  will  be  much  more  detailed.  We  will  use  a  JSEP- 
orlented  modification  of  the  TECEP  approach  (see  Appendix  for  a  description  of 
TECEP).  Table  3  shows  the  cost  categories  for  which  data  will  be  collected. 
Cost  data  will  be  collected  separately  for  PLATO  and  MIcroTICCIT. 
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Table  3 


COST  DATA 


Symbol 

Variable  Description 

Value 

Units. 

Equipment 

EQCISP 

Equipment  costs  Independent  of  the 
number  of  student  positions 

dollars 

L0FEQ1 

Expected  life  of  EQCISP  assets 

7 

years 

EQIMPC 

Equipment  costs  per  student 
position 

dollars 

TSPOSD 

Percent  of  operating  time 
student  position  down 

percent 

C0PMT1 

Operating  and  maintenance  cost  of 
equipment  per  student  position  per  year 

dollars 

0MFEQ1 

Operating  and  maintenance  cost  of 
fixed  equipment  (annual) 

dollars 

SUPPLY  Cost  of  expendable  supplies  per 

student  while  In  course 

Instructional  Material  { IM) 

dollars 

UPDATE 

Percentage  of  original  development 
cost  required  annually  to  maintain 
instructional  material 

percent 

EVIM 

Percentage  of  original  development  cost 
remaining  at  end  of  planning  period 

percent 

CIMD 

Average  cost  of  developing  one  hour 
of  Instructional  material 

dollars 

Personnel 

INTSPO 

Instructor  to  student  position  ratio 

decimal 

SAL I NR 

Annual  salary  and  benefits/instructor 

dollars 

STUDSL  Annual  salary  and  benefits/student 

Ml scellaneous 

dollars 

N 

Number  of  years  in  planning  period 

3 

years 

PRATE 

Pass  rate 

percent 

DRATE 

Discount  rate 

10 

percent 

WSCHOP 

Weeks  school  operates  per  year 

52 

weeks 

TLENGH 

Average  time  spent  In  training 
per  student  per  week 

6.4 

weeks 

TLEGTH 

Average  hours  per  week  student 
spends  In  training 

20 

hours 

Note:  All  percent  values  are  entered  as  decimal  equivalents. 
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The  data  will  be  used  to  calculate  the  following  cost  variables  for  each 
system: 

01.  Total  undiscounted  cost  per  year  - 

Total  annual  operation  A  maintenance  &  equipment  acquisition 
costs  + 

Annual  maintenance  costs  of  Instructional  materials  + 

Total  annual  salary  A  benefits  of  instructors  + 

Total  annual  cost  of  student  supplies  + 

Total  annual  student  salaries  A  benefits 

02.  Total  undiscounted  cost  * 

Sum  of  annual  undiscounted  costs  + 

Total  equipment  acquisition  costs  + 

Total  Implementation  cost  of  instructional  materials  - 
Remaining  value  of  all  facilities,  equipment  A  Instructional 
material  at  end  of  program  (discounted) 

03.  Present  value  « 

Sum  of  discounted  annual  costs  + 

Total  equipment  acquisition  costs  + 

Total  Implementation  costs  of  Instructional  materials  - 
Remaining  value  of  all  facilities,  equipment  A  instructional 
material  at  end  of  program  (discounted) 

04.  Average  discounted  cost  per  student  position  - 
Present  value  / 

Number  of  student  positions 

05.  Average  discounted  cost  per  graduate  - 
Present  value  / 

Number  of  graduates 

06.  Annual  average  undiscounted  cost  per  student  position  ■ 

Total  undiscounted  cost  / 

Number  of  years  of  program  X  number  of  student  positions 

07.  Annual  average  undiscounted  cost  per  graduate  - 
Total  undiscounted  cost  / 

Number  of  years  of  program  X  number  of  graduates 

08.  Annual  average  discounted  cost  per  student  position  « 

Present  value  / 

Number  of  years  of  program  X  number  of  student  positions 

09.  Initial  system  acquisition  cost  * 

Total  facilities  acquisition  costs  + 

Total  equipment  acquisition  costs  + 

Total  Implementation  costs  of  Instructional  materials 

10.  Initial  system  acquisition  cost  per  student  position  - 
Initial  system  acquisition  costs  / 

Number  of  student  positions 
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11.  Average  discounted  cost  per  terminal  hour  ■ 

Total  discounted  cost  / 

Number  of  hours  of  instruction  per  year  X  number  of  years 
of  program 

12.  Average  undiscounted  cost  per  terminal  hour  * 

Total  undiscounted  cost  / 

Number  of  hours  of  instruction  per  year  X  number  of  years 
of  program 

It  is  anticipated  that  these  cost  variables  will  provide  Army  decision¬ 
makers  with  all  of  the  necessary  information  with  which  to  evaluate  JSEP. 

These  cost  variables  include  annual,  average,  and  total  costs  of  the  PLATO  and 
MicroTICCIT  systems. 
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Cost-Effectiveness  Models 


Our  proposed  model  will  incorporate  salient  attributes  gathered  from  the 

bo<fy  of  knowledge  relating  to  cost  models.  Among  these  models  which  undergird 

our  efforts  are  the  following: 

1)  Training  Effectiveness-Cost  Effectiveness  Prediction  (TECEP).  The  Navy's 
technique  for  selecting  cost-effective  instructional  systems  for  proposed 
training  programs.  This  1975  model  serves  as  an  aid  to  a  learning  system 
designer.  Given  a  specific  set  of  training  objectives  and  learning 
algorithms,  one  experienced  in  system  designs  could,  theoretically  choose 
the  cost-effective  instructional  delivery  system  or  combination  of 
systems.  Included  in  the  consideration  is  the  estimated  cost  of  using 
each  alternative  delivery  system  to  obtain  the  required  number  of  students 
to  support  objectives. 

2)  Educational  Technology  Assessment  Model  (ETAM).  A  follow-up  of  TECEP, 
ETaM  was  developed  by  IBM  for  the  Navy.  This  model  is  designed  to  assess 
the  cost/effect  of  a  change  or  Innovation  In  a  course  of  Instruction. 

There  are  several  unique  areas  In  the  model.  For  example,  a  "job/task 
cost  model"  calculates  the  costs  relative  to  errors  in  job  performance. 
ETAM  appears  to  be  a  well -Integrated,  functional,  and  comprehensive 
cost/effective  model.  However,  Its  main  objective  is  to  quantify  the 
impact  of  educational  Innovation  upon  existing  courses. 

3)  MOD I A.  This  Air  Force  model  designed  by  the  RAND  Corporation  appears  to 
be  oriented  toward  new  development  planning.  It  is  essentially  a  cost 
model  structured  for  situations  characteristic  of  the  Air  Force.  At  the 
same  time,  MODIA  has  the  potential  for  exportation  to  other  services  with 
restructuring  of  its  program  budget  and  appropriation  categories. 

4)  Vadhanapanick  Model.  The  doctoral  dissertation  by  Salsawan  Vadhanapanlck 
in  1 9/6  described  a  cost  effectiveness  model  for  application  In  the 
development  of  instructional  delivery  systems.  She  aggregated 
instructional  technology  costs  Into  research  and  development.  Investment, 
and  operating.  Additionally,  she  developed  an  effectiveness  model  which 
considered  a  number  of  learner  variables,  such  as  student  achievement  and 
attitude,  also  Included  were  hardware  variables  such  as  equipment 
capability,  down  time,  and  availability.  Her  cost-effectiveness  model  Is 
an  Integration  of  the  separate  cost  and  effectiveness  models.  The  purpose 
of  the  Integrated  model  Is  to  combine  the  cost  per  student  contact  hour 
and  probability  of  system  effectiveness  into  one  index  for  each  course  of 
action  examined. 

5)  Cost-Effectiveness  Specification  for  Computer  Based  Training-  Systems. 

This  model  was  developed  by  HumRR'tfTor  tne  Army  to  ''facilitate  the 
purchase,  monitoring,  and  evaluation  of  computer  based  training  systems." 
This  1977  model  provides  a  standardized  specification  through  which 
training  system  costs  can  be  discovered.  This  specification  Is  broken 
into  three  distinct  volumes  oriented  toward  training  that  Is  administered, 
aided,  or  managed  by  computer.  The  cost  methodology  concentrates  on  the 
total  Inputs  (personnel,  money,  material)  required  by  a  CBI  system  over 
Its  life  cycle. 


